Using a Hash-Based Method with Transaction Trimming and Database Scan Reduction for Mining Association Rules

In this paper, we examine the issue of mining association rules among items in a large database of sales transactions. Mining association rules means that given a database of sales transactions, to discover all associations among items such that the presence of some items in a transaction will imply the presence of other items in the same transaction. The mining of association rules can be mapped into the problem of discovering large itemsets where a large itemset is a group of items which appear in a suucient number of transactions. The problem of discovering large itemsets can be solved by constructing a candidate set of itemsets rst and then, identifying, within this candidate set, those itemsets that meet the large itemset requirement. Generally this is done iteratively for each large k-itemset in increasing order of k where a large k-itemset is a large itemset with k items. To determine large itemsets from a huge number of candidate large itemsets in early iterations is usually the dominating factor for the overall data mining performance. To address this issue, we develop an eeective algorithm for the candidate set generation. It is a hash based algorithm and is especially eeective for the generation of candidate set for large 2-itemsets. Explicitly, the number of candidate 2-itemsets generated by the proposed algorithm is, in orders of magnitude, smaller than that by previous methods, thus resolving the performance bottleneck. Note that the generation of smaller candidate sets enables us to eeectively trim the transaction database size at a much earlier stage of the iterations, thereby reducing the computational cost for later iterations signiicantly. The advantage of the proposed algorithm also provides us an opportunity of reducing the amount of disk I/O required. Extensive simulation study is conducted to evaluate performance of the proposed algorithm.